Method for a provision of image recordings

ABSTRACT

A method for provision image recordings in a vehicle. The method include: ascertaining a first image recording of a first image sensor of the vehicle and at least one further image recording of at least one further image sensor of the vehicle; carrying out a merging of the first image recording and the at least one further image recording for a reduction of an amount of data for providing the relevant information, the merging taking place based on a selection from the surrounding regions, whereby at least one merged image recording is obtained; initiating at least one transmission of the merged image recording within the vehicle and at least to a central control device of the vehicle, whereby the merged image recording is supplied to a processing for performing the vehicle function.

FIELD

The present invention relates to a method for a provision of imagerecordings. Furthermore, the present invention relates to a computerprogram and to a device for this purpose.

BACKGROUND INFORMATION

Driver assistance systems, parking aids, and highly automated drivingoften use multiple cameras whose images are processed in the centralcontrol device of the vehicle. There are often overlapping regions orparts in the images acquired in this process that do not contain anyadditional information. Examples include reflections of the actual imageof a camera on the hood, recording the sky, and the like. “Region ofinterest” (ROI for short) regions are typically used only in the controldevice, and, in the context of image processing, the remaining data arediscarded if they are not used for the use of additional information.The use of additional information includes, for example, an estimationof the vehicle speed or calibration of the position of the cameras amongthemselves. Data that are unused later are thus transmitted anyway andcause an increase in bandwidth. Nevertheless, it is conventionallyprovided that gateways or zone controllers of the vehicle forward thedata without knowledge of the image content in the vehicle network.

SUMMARY

An object of the present invention is to provide a method, a computerprogram, and an apparatus for providing image recordings. Features anddetails of the present invention are apparent from the disclosureherein. Features and details described in connection with the methodaccording to the present invention of course also apply in connectionwith the computer program according to the present invention as well asthe device according to the present invention, and vice versa in eachcase, so that with regard to the disclosure of the individual aspects ofthe present invention, mutual reference is or can always be made.

In particular, a method according to the present invention is used toprovide image recordings in a vehicle. The vehicle can be for example amotor vehicle and/or an autonomously driving vehicle and/or a truckand/or a passenger vehicle. The image records are in particular datarepresenting at least one image or a sequence of images.

In particular, according to an example embodiment of the presentinvention, the following steps are carried out automatically, preferablyone after the other in the indicated sequence, and the steps can also becarried out repeatedly if necessary:

-   -   ascertaining a first image recording of a first image sensor of        the vehicle and at least one further image recording of at least        one further (e.g. second, third, fourth, etc.) image sensor of        the vehicle, for example by receiving the image recordings from        the image sensors, it being possible for different spatial        regions of a surrounding environment of the vehicle to be imaged        by the image recordings, as a result of which at least one item        of information relevant to a vehicle function of the vehicle is        provided,    -   carrying out a merging of the first image recording and the at        least one further image recording for a reduction of a data        amount for providing the relevant information, wherein the        merging may be carried out based on a selection from the        surrounding regions, whereby at least one merged image recording        is obtained,    -   initiating and/or carrying out at least one transmission of the        merged image recording (at least) within the vehicle and/or at        least to a central control device of the vehicle, whereby the        merged image recording is supplied to processing for performing        the vehicle function.

In particular, a feature of the present invention is that the reductionof the amount of data required to provide the relevant information isupstream of the further processing of the image recordings. To reducethe amount of data, for example a region of interest (ROI for short) ofthe image recordings is defined and selected from the image recordings,i.e., in particular cropped from them. This step is conventionallycarried out by the central control device only during the subsequentprocessing for performing the vehicle function, in particular imageprocessing, but can be separated from it according to the presentinvention.

According to an example embodiment of the present invention, during theascertaining of the image recordings, in addition to the first imagerecording, the other image recordings, e.g. at least a second imagerecording, third image recording, etc., can also be ascertained. Theimage recordings may here have been acquired temporally in parallel bythe different image sensors. The image sensors can be situated atdifferent positions on the vehicle for this purpose. Accordingly,different spatial regions of the surrounding environment of the vehiclecan be imaged (e.g. optically and/or visually) by the image recordings;here the environmental regions may partially overlap and/or containredundant information. For example, a first image sensor may be orientedin the direction of travel (towards the front), and/or a second imagesensor may be oriented in a direction opposite the direction of travel,and/or a third image sensor may be oriented to a left blind spot, and/ora fourth image sensor may be oriented to a right blind spot.

The at least one relevant item of information can include, for example,at least one of the following items of information:

-   -   an item of speed information about a speed of the vehicle,    -   an item of object information about at least one object in the        environment of the vehicle, e.g. at least one vehicle traveling        in front,    -   an item of distance information about a distance to a vehicle        traveling in front.

The at least one vehicle function includes, for example, a driverassistance system and/or a parking assistant and/or highly automateddriving. The image sensor can be a camera, and in particular a stereocamera. The image sensor can also be another device for acquiring thesurrounding environment, such as a radar sensor or the like.Accordingly, the image recordings can each be realized as at leasttwo-dimensional or three-dimensional image data.

According to an example embodiment of the present invention, carryingout the merging can involve a combination of those image recordings thatwere taken temporally in parallel, and thus at substantially the samepoint in time. The combination may be understood such that redundantand/or overlapping regions of the image recordings are at leastpartially removed. Advantageously, the image recordings depict, at leastin part, different regions of the vehicle's surrounding environment thatcontain the relevant information and can therefore be put together likea puzzle. Overlapping and/or redundant regions can be discarded, wherebya selection is made from the surrounding regions of the environment, sothat a reduction in the amount of data in order to provide the samerelevant information is possible. Furthermore, the selection from thesurrounding regions can also be understood to mean that only thoseregions of the image recordings are merged that depict regions of theenvironment that contain the relevant information. This thus correspondsto a restriction of the image recordings to a region of interest.According to a concrete example, for example, portions of the imagerecordings can be removed which are already imaged by another of theimage recordings (overlapping regions), and other portions can beremoved which are not evaluated in the context of the processing and aretherefore not relevant (e.g. a sky or a hood).

The at least one processing for performing the vehicle function mayinclude, for example, an image processing in which the restriction to aregion of interest is also standardly carried out. According to thepresent invention, this restriction can already be carried out beforethe transmission to the central control device and supplying to the atleast one processing, in order in this way to reduce the amount of datato be transmitted.

In addition, in the context of the present invention it can be providedthat for the selection from the surrounding regions, the following stepis carried out:

-   -   removal of at least one region of the image recordings that        redundantly images a portion of the environment and/or an        overlapping region of the environment.

When multiple image sensors, in particular cameras, are used for thevehicle function, the resulting image recordings usually containredundant and possibly even overlapping regions that do not contain anyadditional information. This can be, for example, the hood, which showsreflections of the actual image, or a recorded sky and the like.Usually, such regions are discarded in the central control device ifthey are not used to ascertain additional information. In contrast, itcan be an advantage of the present invention that such regions can beremoved prior to transmission to the central control device. This canreduce the effort and amount of data required in the data transmissionand processing, in particular beyond a level achievable by datacompression.

According to an example embodiment of the present invention, the removedat least one region of one of the image recordings can here redundantlyimage such a portion of the environment and/or an overlapping region ofthe environment that is already imaged by another region of the sameimage recording. The image recording of one (individual) image sensorwould thereby be inherently redundant and/or overlapping. Also, it canbe provided that the removed at least one region of the first imagerecordings redundantly images such a portion of the environment and/oran overlapping environment region that is already imaged by anotherregion of one of the at least one further image recordings. Thus, theimage recording of one of the image sensors would be redundant and/oroverlapping only in combination with one of the further image recordingof another of the image sensors. In both cases, the merging according tothe present invention can reduce the amount of data and still providethe same relevant information.

According to an example embodiment of the present invention, in afurther option, it can be provided that for the selection from thesurrounding regions, the following step is carried out:

-   -   removing at least one region of the image recordings which is        not part of the relevant information.

This removed region can include the portion of the image recordings thatis not part of the region of interest, i.e. a region of the imagerecordings outside of the region of interest. Here this region can be afunction of the position and orientation of the image sensors on thevehicle. For example, the region of the image recordings that depictsthe sky may not contribute to the relevant information and can thereforebe removed.

Optionally, it may be provided that the selection from the surroundingregions is carried out based on at least one item of positioninformation about a position and/or orientation of the image sensors onthe vehicle, the position information being ascertained by at least oneof the following steps:

-   -   evaluating at least one predefined item of information about the        position of the image sensors on the vehicle,    -   carrying out a calibration of the position information, in        particular by a trained algorithm.

In particular, it is a feature of an example embodiment of the presentinvention that based on the knowledge about the installation of thesensors in the vehicle, the step of forming the region of interest canbe separated from the subsequent processing. A use of trained algorithmsfor calibration tracking is also possible here. The merging of the imagerecordings is in particular possible due to the knowledge of themechanical construction and/or the orientation and/or a tracking of thecalibration, e.g. via an artificial intelligence algorithm (e.g. trainedConvolutional Neural Networks, or CNNs). Examples of such algorithms are“Gaffe” or “TensorFlow.” The predefined information can be pre-stored inthe vehicle, for example in a non-volatile data memory.

According to an example embodiment of the present invention, it mayfurther be provided that carrying out the merging includes selecting atleast one region of interest of the image recordings, wherein the regionof interest is or may be defined by the vehicle function, optionallyexcluding regions of the image recordings outside the region of interestfrom the merging and/or transmission. In other words, the region ofinterest can be cropped from the image recordings to form the mergedimage recording therefrom. The region of interest can be defined by thevehicle function, and different vehicle functions may also definedifferent regions-of-interest. For example, some vehicle functionsdefine a surrounding region in the direction of travel as a region ofinterest, whereas other vehicle functions require the surroundingregions in further vehicle directions.

In addition, in the context of the present invention it can be providedthat the carrying out of the merging includes the following step:

-   -   incorporating an item of metadata information into the merged        image record defining at least one region of the image        recordings that is excluded from a transmission.

The inclusion of specific metadata information in the sensor informationmakes it possible for individual regions of the image recordings to beexcluded from transmission at a later time, without the image content ofthe image recordings themselves then having to be processed again.Instead, downstream transmission points (e.g. a control device orgateways) can simply evaluate the metadata information for this purposein order to determine which regions to exclude. In other words, themetadata information can be used to indicate what is located where inthe merged image recording.

According to an example embodiment of the present invention, it is alsooptionally possible that selection information is defined andtransmitted by the central control device, preferably on the basis of atleast one reference marker, and in particular received by a zonecontroller, in order to carry out the merging as a function of theselection information, and in particular in order to adjust the regionof interest. In this way a return channel from the central controldevice is provided through which it is possible to communicate whichregions or portions of the image recording are not used in the furtherprocessing, so that they can no longer be transmitted. The selectioninformation thus forms “feedback” from the control device or processingto the zone controller in order to track the merging and/or region ofinterest. The at least one reference marker in the image recordings canbe used here as orientation for the control device. Thus, the region ofinterest or the calibration can also be adjusted during runningoperation, if necessary. The reference marker is, for example, a markerthat indicates a specific position in the image recording relative to aknown position on the vehicle. Also, if necessary, the reference markercan indicate a scale of the image recordings.

According to an example embodiment of the present invention, it isfurther optionally provided that the at least one piece of informationrelevant to the vehicle function of the vehicle includes an acquisitionvalue, and that carrying out the merging includes at least one of thefollowing steps:

-   -   evaluating at least one acquisition region of the image        recordings to determine therefrom the acquisition value, in        particular a speed of the vehicle, the acquisition region        preferably being different from a region of interest,    -   incorporating additional information into the merged image        recording, the additional information including the detection        value,    -   excluding the acquisition region from the merged image recording        and/or transmission.

A further feature of an example embodiment of the present invention isin particular that the additional information, e.g. about a vehiclespeed or a calibration, can be transmitted as a scalar quantity insteadof via redundant data. The additional information can thus include ascalar quantity, which for example indicates the ascertained detectionvalue for the vehicle speed or another parameter. The evaluation canthus also be realized as a measurement, and the acquisition value can berealized as a measured value. Usually, such a measurement takes placedownstream, for example in the central control device, on the basis ofthe image recordings. For example, the speed can be ascertained byevaluating objects imaged by the image recordings having a fixeddistance from each other, in particular in the course of time. If thisevaluation takes place before transmission to the central controldevice, only the acquisition value has to be transmitted, but not theacquisition region used for evaluation. Thus, the amount of data to betransferred can be significantly reduced. The acquisition region is, forexample, a two-dimensional region of the image recordings or also asequence of images of this region.

According to an advantageous further development of the presentinvention, it can be provided that the merging is carried outdecentrally by a zone controller, such that preferably after theascertaining of the image recordings, the image recordings aretransmitted to the zone controller, the transmission of the merged imagerecording to the central control device preferably being carried out viaa transmission system, in particular bus system, of the vehicle.

The bus system can be, for example, a CAN bus or a vehicle network suchas an Ethernet.

According to an example embodiment of the present invention, it may inaddition be possible for lossy data compression of the image recordingsand/or of the merged image recording to be carried out, where the lossydata compression can include the following steps:

-   -   carrying out a preparation of the respective image recording, in        which a portion of the image recording, in particular a first        data portion, is removed and an artificially generated and        reproducible replacement portion is defined as an approximation        of the removed portion by a replacement item of information in        order to prepare the image recording for a lossless data        compression,    -   carrying out the lossless data compression of the prepared image        recording,        such that preferably during the respective transmission of the        image recordings the replacement information is transmitted        instead of the removed portion. Here, the removed portion and        the replacement portion can each be understood as a portion of        the image recordings, which may represent a portion of each        pixel. For example, the portion is a noise portion. In contrast,        a region, such as the region of interest, cannot be understood        as a portion of individual pixels but rather a selection of        multiple and possibly contiguous pixels. In particular, here        synergies can be achieved by combining carrying out merging        according to the present invention, preferably the formation of        an ROI, with an efficient data compression. For example, in the        zone controller, both the removal of the first data portion and        the removal of regions outside the ROI can be done in a single        step.

In addition, the described data compression yields the advantage thatmore, and more relevant, data can be transmitted, even if the dataconnection used for this purpose is only a connection having lowerbandwidth. The data can here be compressed to a much higher degree thanis possible with many conventional methods. At the same time,falsification of the data can be avoided, since the first data portionis not merely removed, but rather a replacement portion is additionallyprovided as an approximation of the removed portion.

According to an example embodiment of the present invention, thereplacement portion can be artificially generated and/or reproducible.This means that the replacement portion is not obtained from amodification or alteration of the removed portion, but can be generatedcompletely artificially even without knowledge of the removed portion,using the replacement information. For this purpose, a generator such asa random number generator is used, which can generate the replacementportion, e.g. as a matrix or vector of random numbers, on the basis ofthe replacement information, such as a seed. This has the advantage thatthe replacement portion itself does not have to be transmitted, but onlythe replacement information. The replacement information can here beonly a key, such as a seed key, and can thus have a much smaller datasize of a few bits compared to the replacement portion and the removedportion. For example, the replacement information has max. 1% of thedata size of the replacement portion and/or the removed portion.

According to an example embodiment of the present invention, the imagerecordings can be transmitted as data within the vehicle. Datacompression has the advantage that the amount of data to be transmittedcan be reduced. In the following, the image recordings are thereforealso referred to simply as data, for a more detailed description of thedata compression.

Further, within the scope of the present invention, it is possible thatthe removed portion is a noise portion of the data and/or thereplacement portion is an artificially generated noise, in particular apseudo-noise. The removed portion can be a particular portion of thedata, which can be characterized and selected according to predefinedand, in particular, statistical criteria. When the first data portion isremoved, the prepared data may include only the portion of the dataremaining after the removal. (In the context of the present invention,the removed portion is also referred to as the first data portion, andthe remaining portion as the second data portion.) The remaining portioncan have the relevant information content, e.g. a recording of anenvironment of the vehicle. In contrast, the removed portion can have alower and predominantly random information content, e.g. as noiserepresenting merely statistically distributed energy.

It can be provided within the scope of an example embodiment of thepresent invention that the data compressed by the lossless datacompression as well as the replacement portion are reproduced inbit-identical fashion on the basis of the replacement information afterthe transmission. However, in combination with the preparation, a lossydata compression of the data may still result, because here the dataportion is removed and the lossless data compression is applied only tothe remaining portion of the data. “Bit-identical” means in particularthat each bit of the data can be reproduced identically and thus withoutloss.

It is possible that through a removal of the first data portion, inparticular the noise portion, of the data, a much higher compression canbe achieved than would be the case with conventional methods. Since theremoved data portion can have a rather random and statisticallydistributed information content, only an inefficient compression wouldbe possible. Due to the predominantly random content of the removedportion, this portion can be statistically approximated instead, thusenabling a substitution by the artificially generated replacementportion. In contrast, the data portion remaining after the removalcannot be statistically replicated due to the predominantly non-randominformation content, but can nevertheless be better compressed due tothe extensive redundant information. In other words, the determined datamay consist of a random first data portion, especially noise, and asecond data portion with a high proportion of redundant information. Ifthe data are in the form of image data, the first data portion can bethe (superimposed) noise portion of the image, and the second dataportion can be the remaining portion of the image after subtracting thenoise. This can also hold for each pixel of the image; this can be madeup of the first (random, noise) and second (redundant) data portion. Thein particular lossless data compression can be applied to the second(redundant) data portion. The first data portion, on the other hand, canbe replaced by the replacement portion, in particular based on a noisemodel. Since the replacement portion does not have to be transmitted,but can be reproduced bit-identically using the replacement information,the data size to be transmitted is reduced. Although the replacementportion (only) approximately corresponds to the first data portion, theoriginal data portion is irretrievably lost, so that this procedure canalso be understood as lossy data compression. In other words, this lossydata compression for the first data portion is combined with thelossless data compression for the second data portion. This allows acompressed quantity of data to be transmitted.

Since the first data portion is removed, and moreover is notreproducible, the method according to an example embodiment of thepresent invention may include a lossy data compression, in which thefirst data portion is lost but the remaining second data portion may becompressed losslessly. To avoid falsification of the data, a replacementportion can be generated instead of the first data portion. Aftertransmission, the replacement portion can be reproduced without loss, inparticular bit-identically, and reimpressed on the transmitted data. Forexample, a reproducible pseudo-noise can be used as the replacementportion. Here, although the replacement portion does not replace theremoved data portion bit-identically, it does come very close to it.Through this impressing of reproducible pseudo-noise, a strongcompression factor can be achieved. The remaining portion of the data aswell as the replacement portion can be reproduced bit-identically afterthe transmission. If the removed first data portion is a noise portionof the data, replacing the noise portion with the replacement portioncan also be referred to as a noise substitution. Noise substitutionallows the greatest portion of the random noise in the data, which canbe characterized by an input noise model, to be replaced by thepseudo-noise, which is characterized by a target noise model.

In addition, it is possible within the scope of an example embodiment ofthe present invention for the replacement portion to be defined byspecifying the replacement information, in particular a seed key for agenerator such as a random number generator, the replacement portionpreferably being subsequently reproduced by the replacement information,in particular after the transmission of the data. The replacementinformation thus unambiguously defines the replacement portion, e.g. thepseudo-noise, so that the replacement portion can be reproducedbit-identically on the basis of the replacement information. For thispurpose, for example the replacement information is used by a generator,or random number generator, to generate the replacement portion in theform of a two-dimensional matrix. The seed key can be determined, forexample, by measuring the noise characteristic of the (real) imagesensor. The statistical characteristic of this pseudo-noise thuscorresponds to that which really occurred in the image sensor, but isbit-identically reproducible.

Furthermore, it is possible within the scope of an example embodiment ofthe present invention that carrying out the preparation includes atleast one of the following steps:

-   -   incorporating at least one piece of additional information into        the image recording, which preferably includes an acquisition        value,    -   incorporating metadata information into the image recording,    -   incorporating at least one reference point, in particular        reference marker, which is designed to determine a position of        the image sensors and/or the positions of the image sensors        relative to one another.

Thus, for example, a use of a lossy data compression is provided inwhich in addition at least one reference point can be embedded in theimage recording in order to enable, via the reference point, efficientdetermination of the position of the image sensors relative to oneanother. The at least one reference point, preferably a plurality ofreference points, can here be specific to the position and/ororientation of the image sensors, and in particular can also be used tocalibrate the position information.

The subject matter of the present invention also includes a computerprogram, in particular a computer program product, having instructionsthat, when the computer program is executed by a computer, cause thecomputer to carry out the method according to the present invention.Thus, the computer program according to the present invention providesthe same advantages as described in detail above in relation to a methodaccording to the present invention.

The subject matter of the present invention also includes a device fordata processing. As the computer, for example the data processing devicethat executes the computer program and/or the method according to thepresent invention can be provided. The device and preferably thecomputer can include at least one processor for executing the computerprogram. Also, a non-volatile data memory can be provided in which thecomputer program can be stored and from which the computer program canbe read out by the processor for execution. The device according to thepresent invention can also have multiple processors and/or be designedas a computer system. For example, the device according to the presentinvention can include a central control device of the vehicle and/or oneor more decentralized zone controllers of the vehicle. Also, the deviceaccording to the present invention can optionally designate the entirevehicle electronics.

The subject matter of the present invention can also be acomputer-readable storage medium having the computer program accordingto the present invention. The storage medium is designed, for example,as a data storage device such as a hard disk and/or a non-volatilememory and/or a memory card. The storage medium can be integrated intothe computer, for example.

Furthermore, the method according to the present invention can also berealized as a computer-implemented method.

Further advantages, features, and details of the present inventionresult from the following description, in which exemplary embodiments ofthe present invention are described in detail with reference to thefigures. The features mentioned in the description can be essential tothe present invention individually or in any combination.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a visualization of the steps of a method according to anexample embodiment of the present invention.

FIG. 2 shows a further visualization of the steps of a method accordingto an example embodiment of the present invention,

FIG. 3 shows a visualization of the steps of a method according to anexample embodiment of the present invention with further details.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

In the following figures, identical reference signs are used for thesame technical features even of different exemplary embodiments.

FIG. 1 schematically illustrates the steps of a method 100 according tothe present invention. The method is used here for a provision of imagerecordings 271, 272 in a vehicle 1, the image recordings preferablybeing acquired by a plurality of image sensors 11, 12. The image sensors11, 12 are in particular cameras, and thus imagers each with a chip withdefined resolution, frame rate, and noise characteristics. A first imagesensor 11 and a second image sensor 12 on vehicle 1 are shown asexamples. According to their configuration and orientation, imagesensors 11, 12 can detect different spatial environmental regions 301,302 in a surrounding environment 3 of the vehicle 1.

According to a first method step, an ascertaining 101 of a first imagerecording 271 of the first image sensor 11 of the vehicle 1 and at leastone further image recording 272 of at least one further image sensor 12of the vehicle 1 takes place.

Corresponding to the configuration and orientation of the image sensors11, 12, the different surrounding spatial environmental regions 301, 302can be imaged by the image recordings 271, 272. This imaging isnecessary in order to provide at least one item of information relevantto a vehicle function of vehicle 1. For example, the image recordings271, 272 can be used for detection of objects in the environment 3 by adriver assistance system.

To improve a transmission 130 of the relevant information within vehicle1, a merging 410 of the first image recording 271 and the at least onefurther image recording 272 is then carried out. This can achieve areduction of a data volume for the provision of the relevantinformation. Here the merging 410 can take place based on a selection415 from the environmental regions 301, 302, whereby at least one mergedimage recording 416 is obtained. For example, the selection 415 iscarried out based on a predefined region of interest 460, which may bedefined as a function of the vehicle function.

Subsequently, the at least one transmission 130 of the merged imagerecording 416 can be initiated within vehicle 1. The merged imagerecording 416 is transmitted to, for example, a central control device310 of the vehicle 1, whereby the merged image recording 416 can besupplied to a processing 170 for performing the vehicle function. Thecentral control device 310 can for example carry out the vehiclefunction through the processing 170.

In addition, according to FIG. 1 , the merging 410 can be carried out ina decentralized manner by a zone controller 320 visualized in FIG. 2 ,the image recordings 271, 272 preferably being transmitted to zonecontroller 320 after the ascertaining 101 of the image recordings 271,272. Preferably, the transmission 130 of merged image recording 416 tocentral control device 310 can be carried out via a transmission system330, in particular a bus system, of vehicle 1. In addition, the at leastone item of information relevant for the vehicle function of vehicle 1can include an acquisition value 420, and the carrying out of themerging 410 can include the following steps:

-   -   evaluating at least one acquisition region 430 of the image        recordings 271, 272 to determine therefrom the acquisition value        420, in particular a speed of the vehicle 1,    -   incorporating an item of additional information 260 that        includes the acquisition value 420 into the merged image        recording 416,    -   excluding the acquisition region 430 from the merged image        recording 416 and/or the transmission 130.

The selection 415 from the surrounding regions 301, 302 can in additionbe carried out based on at least one item of position information 470about a position and/or orientation of the image sensors 11, 12 onvehicle 1, the position information 470 being ascertained by at leastone of the following steps:

-   -   evaluating at least one predefined item of information about the        position of the image sensors 11, 12 on vehicle 1,    -   carrying out a calibration of the position information 470.

FIG. 2 further illustrates that an item of selection information 450 isdefined and transmitted by the central control device 310 based on atleast one reference marker 480, and in particular received by a zonecontroller 320 in order to perform the merging 410 as a function of theselection information 450.

The zone controller 320 can be designed to bundle the image recordings271, 272 and possibly further data from additional image sensors 11, 12and to transmit them further to central control device 310. Therefore,the reduction of the image recordings 271, 272 visualized in FIG. 1 canbe done by zone controller 320. In particular, for this purpose, in thecarrying out of the merging 410 according to FIG. 1 , redundant data ofthe image recordings 271, 272 is removed and implicitly containedinformation such as speed of vehicle 1 is extracted as acquisition value420 and explicitly added to merged image recording 416. A transmissionwith a smaller amount of data is thus possible. The reference marker 480visualized in FIG. 2 and the metadata information 440 visualized in FIG.1 can be used to obtain information which of the data from imagerecordings 271, 272 is to be used in the subsequent processing 170.

Carrying out the preparation 110 may further include, according to FIG.1 , incorporating at least one item of additional information 260 intothe image recording 271, 272, such that the additional information 260can include a detection value 420. In addition, an incorporation ofmetadata information 440 into the image recording 271, 272 can beprovided. Moreover, it is possible for an incorporation to take place ofat least one reference point 485 that is designed to determine aposition of the image sensors 10 and/or the positions of the imagesensors relative to one another. This reference point 485, preferably aplurality of reference points 485, can be directly embedded in the imagerecordings 271, 272. In this way, the amount of data to be transmittedcan be readjusted.

FIG. 2 shows that a reference marker ascertaining can take place at 180.Here, in particular in combination with a data decompression, referencemarker 480 is extracted from image recordings 271, 272. At 185, anevaluation of the reference marker 480 can subsequently take place todetermine, for example, further information, such as an exact positionof the imaged surrounding region 301, 302, from the image recordings271, 272.

FIG. 3 visualizes further optional steps of a method 100 according tothe present invention for a provision of data 210, i.e. image recordings271, 272. As an example, an image sensor is shown that can include achip with defined resolution, frame rate, and noise characteristics. InFIG. 3 , image sensor also stands in representative fashion for aplurality of image sensors 11, 12. Digital images ascertained by suchimage sensors 10 typically contain a high degree of entropy, due inlarge part to sensor noise. The presence of this noise is an importantaspect of what gives the image its natural statistical properties. Aremoval of this noise can cause processing algorithms applied to theimages, e.g. in the context of a further processing 170, to providedifferent results than for an image that still contains this noise. Data210 would thus be falsified. However, the presence of the noise stronglylimits the data reduction achievable by a lossless compression.

In the following, it is described in more detail how higher compressionrates can be achieved through the use of a replacement portion 230 whileat the same time maintaining a realistic noise profile, so that contentfalsification can be avoided.

Image sensors 10 can be part of vehicle 1 and can perform an acquisition140 in an environment 3 of vehicle 1 or at vehicle 1 itself. Data 210can be ascertained based on the acquisition 140 and can be for examplesensor data outputted by image sensors i.e. image recordings 271, 272described above. Acquisition 140 can include a recording of contentrelevant to the vehicle function, in particular of objects in thesurrounding environment 3 of vehicle 1. It can further be possible tocarry out a vehicle function through a processing 170 of the data 210.

According to a first method step, the ascertaining 101 of the data 210is carried out at the vehicle 1. The data can be temporarily stored, forexample, after the acquisition 140, in order to transmit it at a latertime when a data connection 2 is available. For this purpose, it can bepossible to repeatedly detect whether data connection 2 is available inorder to then initiate the process steps. For the transmission 130, apreparation 110 of the ascertained data 210 can be carried out, in whicha portion 220 of the data 210 is removed and an artificially generatedand reproducible replacement portion 230 is defined, as an approximationof the removed portion 220, by an item of replacement information 231,in order to prepare the data 210 for an in particular lossless datacompression 120. Here, after removing portion 220, data 210 may stillinclude a remaining portion 240 that is particularly suitable for thelossless data compression 120. The removed portion 220 may also bereferred to as the first data portion 220 and the remaining portion 240may be referred to as second data portion 240.

The step of preparing 110 can be carried out at least in part by agenerator 20. Further, in this step an addition of further informationto the data 210 can optionally take place, for example an item ofadditional information 260 and/or a watermark 250. Subsequently, datacompression 120 of the prepared data 210 can be carried out andtransmission 130 of the data 211 compressed by the data compression 120can be initiated and/or carried out via the data connection 2. Here theremoved portion 220 is not transmitted; rather, instead of portion 220,replacement information 231 is transmitted together with the remainingportion 240. Here, removed portion 220 can be a noise portion 220 of thedata 210 and replacement portion 230 can be an artificially generatednoise 230, in particular a pseudo-noise 230.

The replacement portion 230 can be defined during the preparation 110 bydefining the item of replacement information 231, in particular a seedkey for the generator 20 or for a random number generator 20. It is thenpossible to use the replacement information 231 to generate and thusreproduce the replacement portion 230, in particular after transmission130 and decompression 150 of the data 210. For the definition and/orgeneration 160 of the replacement portion 230, it can also be providedthat the replacement portion 230 is generated in the form of apseudo-noise 230 by an application of the generator 20. To enable abit-identical reproduction of the replacement portion 230, the generator20 can be carried out with defined initial conditions, in particular asa function of a target noise model, referred to as a noise model forshort. The initial conditions may be specified and transmitted by theitem of replacement information 231.

In addition, FIG. 3 shows a computer program 40 according to the presentinvention for carrying out the method steps, and a data processingdevice 50 according to the present invention.

The removal of portion 220 can include removing most of the noise fromthe data 210, in particular the image data 210, for example by noisereduction. If necessary, corrections can also be made here, such as ofsensor-specific noise, noise due to fixed patterns, or non-uniformphotosensitivity. After this step, the prepared data 210 are obtained,which can also be referred to as noise-reduced data 210.

Optionally, the generated pseudo-noise 230 can be added to thenoise-reduced data 210, particularly image data 210, at a later time,such that the resulting pseudo-noise image accurately mimics the desirednoise model. The desired noise model can be suitable to mimic therelevant image sensor 10.

The prepared data 210 can be compressed and transmitted without loss. Inthis context, the data 210 can optionally include, in addition to a datacontent, the noise model with the at least one corresponding parameter(i.e. in particular replacement information 231, or the seed key), andpreferably metadata and/or further additional information 260. The atleast one parameter can be stored, e.g. using a steganographic key, inthe data 210 itself or in further data or in a separate file. Losslesscompression can take place for example by a factor of 5 to 10,preferably using a lossless codec defined in the JPEG2000 standard or auser-defined lossless codec. Examples are lossless JPEG or PNGcompression, or also a ZIP compression. The at least one parameter ofthe pseudo-noise can optionally be stored along with the compressed data210. The decompression 150 can take place by first decompressing thedata 210 using the same lossless codec, and then generating thepseudo-noise based on the at least one parameter and adding it to thedata 210.

In the following, an exemplary embodiment of the method steps accordingto the present invention are described in more detail. Here theascertained data 210 can be in the form of image data 210 that have aplurality of pixels i with the respective values x_(i) have. Here eachpixel i can have the first data portion 220, i.e. the noise portion 220,and the remaining second data portion 240 having partly redundantinformation. This means that the two data portions 220, 240 can overlap.The removal of first data portion 220 as part of the preparation 110 ofdata 210 can take place for example by noise reduction. Conventionalnoise reduction techniques can be used for noise reduction. A noisemodel is usually used for this purpose. The prepared data 210 can thenstill include only the remaining portion 240 and the thus noise-reduceddata 210. For example, a Poisson-Gaussian model can be used as a noisemodel, for which the estimated standard deviation σ_(i) of the pixel iwith the value x_(i) is given by σ_(i)=√{square root over(a(x_(i)−x₀)+b)}. Here the noise parameters for this model are a. Thesecorrelate in particular with the signal amplification in image sensor10. The black level of image sensor 10 can be specified by x₀.Furthermore, b can denote a parameter related to the read-out noise ofimage sensor 10. While this noise model is preferably suitable for CCDand CMOS raw image data, a simplified model can also be used in which itis assumed that the noise has a standard deviation σ₀ which isindependent of the pixel value. In this case, the number of noise bitsper pixel i for image data 210 can be calculated with integer values, asN_(Bits)=log₂(σ₀√{square root over (12)})=log₂ (σ₀)+1,792. This numbercan be e.g. 6 to 8 bits. Using the noise reduction techniques, it isthen possible to ascertain, for the respective values x_(i) of thepixels i, a noise-reduced value y_(i). For example, a pseudo-randomnumber generator 20 with a seed S is used for this purpose, where S canbe an integer, in order to generate for each pixel i a pseudo-randomnumber R_(i). The noise-reduced pixel values y_(i) of the noise-reduceddata 210 can be calculated, as an example, by:

$y_{i} = {{{round}\left( {\frac{x_{i}}{\frac{\sigma_{0}}{q}} + R_{i}} \right)}.}$

In the generation of the noise-reduced data 210 using the noisereduction techniques, first data portion 220 is removed. Accordingly,first data portion 220 can designate the difference between theoriginally ascertained data 210 and the prepared noise-reduced data 210.Here first data portion 220 can include most of the natural noise in theascertained data 210.

After the removal of portion 220, the replacement portion 230 can bedefined. Replacement portion 230 can be a pseudo-noise 230, which shouldbe as close as possible to the removed noise. Replacement portion 230can be generated based on replacement information 23, in the presentexample a seed key. A target noise model can be used for this purpose.In the simplest case, the seed key can be arbitrarily predefined forthis target noise model and can be fixedly stored for the methodaccording to the present invention. Thus, defining replacement portion230 does not require an additional calculation step. Alternatively, theseed key can also be calculated based on the removed data portion 220,e.g. using an optimization method to evaluate noise in the removedportion 220. Such a seed key can then be defined, which results inreplacement portion 230 being as close as possible to removed portion220. Further, the seed key can also be defined based on the noisecharacteristic of acquisition device 10, in such a way that replacementportion 230 generated therefrom is as close as possible to removedportion 220. For this purpose, the seed key can also be determinedempirically or in model-based fashion, if necessary.

Remaining portion 240 can then be compressed, in particular losslesslycompressed, and replacement information 23 can be transmitted toreceiver 30 together with the compressed remaining portion 240.

After transmission 130, a decompression 150 of remaining portion 240 canfirst take place in order to obtain decompressed data 212. Subsequently,using the transmitted replacement information 231, pseudo-noise 230 canbe reimpressed on remaining portion 240, for example through thefollowing calculation of the pixel values

$z_{i} = {{{round}\left( {\left\lbrack {y_{i} - R_{i}} \right\rbrack \cdot \frac{\sigma_{0}}{q}} \right)}.}$

A noise model and its parameters are used here, the noise model beingthe functional form of the transformation and the parameters being theexact values for σ₀ and q. The pseudo-random number R_(i) can becalculated by generator 20, which receives the transmitted replacementinformation 231 for the random number generation for this purpose. Data210 obtained in this manner, with the pixel values z_(i), are very closeto the original ascertained data 210 and can subsequently be transmittedto another device 60 for further processing 170.

The above explanation of the specific embodiments describes the presentinvention by way of example only. Of course, individual features of theembodiments can be freely combined with each other, if this makes sensetechnically, without departing from the scope of the present invention.

1-13. (canceled)
 14. A method for a provision of image recordings in avehicle, the method comprising the following steps carried out inautomated fashion: ascertaining a first image recording of a first imagesensor of the vehicle, and at least one further image recording of atleast one further image sensor of the vehicle, different spatialenvironmental regions of an environment of the vehicle being imaged bythe first and further image recordings, to provide at least one item ofinformation relevant for a vehicle function of the vehicle; carrying outa merging of the first image recording and the at least one furtherimage recording for a reduction of an amount of data for providing therelevant information, the merging taking place based on a selection fromthe environmental regions, to obtain at least one merged imagerecording; and initiating at least one transmission of the merged imagerecording within the vehicle and at least to a central control device ofthe vehicle, whereby the merged image recording is supplied to aprocessing for performing the vehicle function.
 15. The method asrecited in claim 14, wherein, for the selection from the environmentalregions, the following step is carried out: removing at least one regionof the first and further image recordings that redundantly images aportion of the environment and/or an overlapping region of theenvironment.
 16. The method as recited in claim 14, wherein, for theselection from the environmental regions, the following step is carriedout: removing at least one region of the first and further imagerecordings that is not part of the relevant information.
 17. The methodas recited in claim 14, wherein the selection from the surroundingregions is carried out based on at least one item of positioninformation about a position and/or orientation of the first and furtherimage sensors on the vehicle, the position information being ascertainedby performing at least one of the following steps: evaluating at leastone predefined item of information about the position of the first andfurther image sensors on the vehicle, carrying out a calibration of theposition information by a trained algorithm.
 18. The method as recitedin claim 14, wherein the carrying out of the merging includes selectingat least one region of interest of the first and further imagerecordings, the region of interest being defined by the vehiclefunction, regions of the first and further image recordings outside theregion of interest being excluded from the merging and/or thetransmission.
 19. The method as recited in claim 14, wherein thecarrying out of the merging includes the following step: incorporatinginto the merged image recording an item of metadata information definingat least one region of the first and further image recordings that isexcluded from a transmission.
 20. The method as recited in claim 18,wherein an item of selection information is defined and transmitted bythe central control device based on at least one reference marker, andis received by a zone controller, in order to carry out the merging as afunction of the selection information to adjust the region of interest.21. The method as recited in claim 14, wherein the at least one item ofinformation relevant for the vehicle function of the vehicle includes anacquisition value, and the carrying out of the merging includes thefollowing steps: evaluating at least one acquisition region of the firstand further image recordings to determine therefrom the acquisitionvalue, the acquisition value including a speed of the vehicle, theacquisition region being different from a region of interest;incorporating an item of additional information that includes theacquisition value into the merged image recording; and excluding theacquisition region from the merged image recording and/or thetransmission.
 22. The method as recited in claim 14, wherein thecarrying out of the merging is accomplished in decentralized fashion bya zone controller, the first and further image recordings beingtransmitted to the zone controller after the ascertaining of the firstand further image recordings, the transmission of the merged imagerecording to the central control device being carried out via atransmission system including a bus system of the vehicle.
 23. Themethod as recited in claim 14, wherein a lossy data compression of arespective image recording of the image first and further imagerecordings and/or the merged image recording is carried out, the lossydata compression including the following steps: carrying out apreparation of the respective image recording, in which a portion of therespective image recording is removed and an artificially generated andreproducible replacement portion is defined, as an approximation of theremoved portion, by an item of replacement information, in order toprepare the respective image recording for a lossless data compression;carrying out the lossless data compression of the prepared imagerecording, the replacement information being transmitted instead of theremoved portion during a respective transmission of the respective imagerecording.
 24. The method as recited in claim 23, wherein the carryingout of the preparation includes at least one of the following steps:incorporating at least one item of additional information including anacquisition value, into the respective image recording, incorporatingmetadata information into the respective image recording, incorporatingat least one reference point, which is configured to determine aposition of the first and further image sensors and/or the positions ofthe first and further image sensors relative to one another.
 25. Anon-transitory computer-readable medium on which is stored a computerprogram including instructions for a provision of image recordings in avehicle, the instructions, when executed by a computer, causing thecomputer to perform the following steps carried out in automatedfashion: ascertaining a first image recording of a first image sensor ofthe vehicle, and at least one further image recording of at least onefurther image sensor of the vehicle, different spatial environmentalregions of an environment of the vehicle being imaged by the first andfurther image recordings, to provide at least one item of informationrelevant for a vehicle function of the vehicle; carrying out a mergingof the first image recording and the at least one further imagerecording for a reduction of an amount of data for providing therelevant information, the merging taking place based on a selection fromthe environmental regions, to obtain at least one merged imagerecording; and initiating at least one transmission of the merged imagerecording within the vehicle and at least to a central control device ofthe vehicle, whereby the merged image recording is supplied to aprocessing for performing the vehicle function.
 26. A device for dataprocessing, configured to provide image recordings in a vehicle, thedevice configured to, in automated fashion: ascertain a first imagerecording of a first image sensor of the vehicle, and at least onefurther image recording of at least one further image sensor of thevehicle, different spatial environmental regions of an environment ofthe vehicle being imaged by the first and further image recordings, toprovide at least one item of information relevant for a vehicle functionof the vehicle; carry out a merging of the first image recording and theat least one further image recording for a reduction of an amount ofdata for providing the relevant information, the merging taking placebased on a selection from the environmental regions, to obtain at leastone merged image recording; and initiate at least one transmission ofthe merged image recording within the vehicle and at least to a centralcontrol device of the vehicle, whereby the merged image recording issupplied to a processing for performing the vehicle function.